Endoscope system

ABSTRACT

An endoscope system includes: a generating means generating a compositing mask that serves as compositing ratios of the corresponding pixels between a pair of images acquired by simultaneously imaging two optical images having different focus positions, into which a subject image is divided on the basis of the ratios of contrasts; a correcting means subjecting compositing masks generated for pairs of images acquired in time series, to weighted averaging for respective pixels, thus generating a corrected mask; and an compositing means compositing the two images according to the corrected mask. The correcting means subjects the compositing masks to weighted averaging by performing weighting such that the percentage of the past compositing masks is higher at pixels constituting a static area and an area having contrast lower than a threshold than at pixels constituting a moving-object area or an area having contrast equal to or higher than the threshold.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of International Application PCT/JP2015/075606which is hereby incorporated by reference herein in its entirety.

This application is based on Japanese Patent Application No.2014-190167, the contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to an endoscope system, and, inparticular, to an endoscope system with extended depth of field.

BACKGROUND ART

In general, it is known that, in devices having an image acquisitionelement, such as endoscope systems, the depth of field is narrowed asthe number of pixels in the image acquisition element becomes higher.Specifically, in the image acquisition element, when the pixel pitch(the horizontal and vertical size of one pixel) is reduced in order toincrease the number of pixels, the permissible circle of confusion isaccordingly reduced, and thus the depth of field of an image-acquisitiondevice is narrowed.

Thus, there is a known endoscope in which one optical image from anobjective lens is divided into two optical images having different focuspositions by a polarizing beam splitter, one optical image is made topass via a λ/4 wavelength plate and a mirror, the other optical image ismade to pass via a mirror, the optical images are imaged on an imageacquisition element, and the two images are composited, therebyacquiring an image in which the depth is extended (for example, see PTL1 and PTL 2).

More specifically, PTL 1 and PTL 2 describe that a polarizing beamsplitter is used as an optical-path dividing element, a depolarizingplate or a λ/4 wavelength plate is provided to suppress a luminanceloss, two images are imaged on one image acquisition element to achievea reduction in size, and a difference between the two images iscorrected when the two images are composited. In particular, PTL 1describes that the depolarizing plate is disposed between the polarizingbeam splitter and the objective lens to convert light in the polarizedstate entering the polarizing beam splitter into circularly polarizedlight, thus achieving uniform polarized-light separation.

Furthermore, when there is a difference in luminance between two images,that is, far and near images, to be composited, luminance unevenness iscaused in a spatial direction or in a temporal direction in the imageobtained after compositing; therefore, in order to resolve this, PTL 3describes that a subject is subjected to matching in the temporaldirection to smooth the image signal, thereby suppressing luminanceunevenness in the composited image.

CITATION LIST Patent Literature {PTL 1} Japanese Unexamined PatentApplication, Publication No. 2013-244104 {PTL 2} Publication of JapanesePatent No. 5393926 {PTL 3} Japanese Unexamined Patent Application,Publication No. 2010-187207 SUMMARY OF INVENTION Solution to Problem

According to one aspect, the present invention provides an endoscopesystem including: an objective optical system that is provided at adistal end of an insertion portion and that obtains a subject image of asubject irradiated with illumination light from a light source; anoptical-path dividing means that divides the subject image into twooptical images having different focus positions; an image acquisitionelement that simultaneously images the two optical images havingdifferent focus positions to acquire a pair of images; a contrastcalculating means that calculates contrasts, for respective pixels, ofthe pair of images acquired by the image acquisition element; a maskgenerating means that generates a compositing mask that serves ascompositing ratios of the corresponding pixels between the pair ofimages on the basis of the ratios of the contrasts calculated by thecontrast calculating means; a mask correcting means that generates acorrected mask by applying, for the respective pixels, weightedaveraging to a plurality of compositing masks that are generated in timeseries by the mask generating means for a plurality of pairs of imagesthat are acquired in time series by the image acquisition element; andan image compositing means that composites the two images according tothe corrected mask generated by the mask correcting means, wherein themask correcting means applies weighted averaging to the plurality ofcompositing masks by performing weighting such that the percentage ofthe past compositing masks is higher at pixels that constitute a staticarea and an area having contrast lower than a predetermined threshold,in the pair of images, than at pixels that constitute a moving-objectarea or an area having contrast equal to or higher than thepredetermined threshold, in the pair of images.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a view showing, in outline, the configuration of an endoscopesystem according to an embodiment of the present invention.

FIG. 2 is a view showing, in outline, the configuration of an imageacquisition unit used in the endoscope system according to theembodiment of the present invention.

FIG. 3 is a view showing, in outline, the configuration of an imageacquisition element used in the endoscope system according to theembodiment of the present invention.

FIG. 4A is a view showing, in outline, the configuration of an exampledepolarizing plate used in the endoscope system according to theembodiment of the present invention.

FIG. 4B is a view showing, in outline, the configuration of an exampledepolarizing plate used in the endoscope system according to theembodiment of the present invention.

FIG. 4C is a view showing, in outline, the configuration of an exampledepolarizing plate used in the endoscope system according to theembodiment of the present invention.

FIG. 5A is a graph showing the relationship between a depolarizing plateand the intensities of a pair of images.

FIG. 5B is a graph showing the relationship between a depolarizing plateand the intensities of the pair of images.

FIG. 6 is a graph showing wavelength dispersion properties when adepolarizing plate and a λ/4 wavelength plate have inverse dispersionproperties.

FIG. 7 is an intensity distribution graph of a pair of images when azero-order wavelength plate that gives a λ/4 phase difference at awavelength of 550 nm is used.

FIG. 8 is a block diagram showing, in outline, the configuration of animage-compositing processing unit in the endoscope system according tothe embodiment of the present invention.

FIG. 9 is a graph of the spectral characteristics of a polarizationseparation coating used for a polarizing beam splitter, showing thespectral characteristics on the optical axis.

FIG. 10 is a graph of the spectral characteristics of a polarizationseparation coating used for the polarizing beam splitter, showing thespectral characteristics obtained when a light beam inclined at an angleof −7 degrees with respect to the optical axis is incident on thepolarization separation coating.

FIG. 11 is a graph of the spectral characteristics of a polarizationseparation coating used for the polarizing beam splitter, showing thespectral characteristics obtained when a light beam inclined at an angleof +7 degrees with respect to the optical axis is incident on thepolarization separation coating.

FIG. 12 is a flowchart showing the flow of compositing two images in theendoscope system according to the embodiment of the present invention.

FIG. 13 is a graph showing example weighting performed in a case inwhich past compositing masks are subjected to weighted averaging, in theendoscope system according to the embodiment of the present invention.

FIG. 14 is a view for explaining a case in which past compositing masksto be subjected to weighted averaging are sequentially changed in timeseries, in the endoscope system according to the embodiment of thepresent invention.

DESCRIPTION OF EMBODIMENTS

An embodiment of the present invention will be described below withreference to the drawings.

As shown in FIG. 1, an endoscope system 1 of the embodiment of thepresent invention includes: an endoscope 2; a light source device 3 thatsupplies illumination light to the endoscope 2; a processor device 4that subjects an image signal obtained by an image acquisition elementprovided in the endoscope 2 to image processing; and an image displaydevice 5 that displays, as an endoscope image, the image signal that hasbeen subjected to predetermined image processing by the processor device4.

The endoscope 2 has: an elongated insertion portion 6 that is insertedinto a body to be examined; an operating portion 7 that is provided at arear end of the insertion portion 6; and a first cable 8 that extendsfrom the operating portion 7. A light guide 9 for transmittingillumination light is inserted through the first cable 8. Anillumination lens 15 that spreads illumination light emitted from thelight guide 9, an objective optical system 16 that obtains a subjectimage, and an image acquisition unit 19 that acquires the subject imageare provided in a distal-end portion 6 a of the insertion portion 6 ofthe endoscope 2. A light-guide connector 8 a provided at an end portionof the first cable 8 is detachably connected to the light source device3 such that a rear-end portion of the light guide 9, which is insertedthrough the first cable 8, serves as an entrance end of the illuminationlight.

The light source device 3 includes, as a light source, a lamp 11, e.g.,a xenon lamp or the like. Note that the light source is not limited tothe lamp 11, such as a xenon lamp, and a light-emitting diode(hereinafter, referred to as “LED”) may be used. The passing light levelof the white light produced in the lamp 11 is adjusted by a diaphragm12, and the white light is then condensed by a condenser lens 13 and isincident on (supplied to) an entrance-end surface of the light guide 9.Note that the degree of opening of the diaphragm 12 can be changed by adiaphragm driving unit 14.

The light guide 9 guides illumination light entering the entrance end(rear end) thereof from the light source device 3 toward the distal-endportion 6 a of the insertion portion 6. The illumination light guided tothe distal-end portion 6 a is spread out from an exit end (distal end)of the light guide 9 by the illumination lens 15, which is disposed on adistal-end surface of the distal-end portion 6 a, and is emitted via anillumination window 15 a, thus illuminating an observation target sitein the body to be examined.

Light from the observation target site illuminated with the illuminationlight is collected by the objective optical system 16, which is mountedon an observation window 20 that is provided adjacent to theillumination window 15 a on the distal-end portion 6 a.

The objective optical system 16 is provided with: an optical elementgroup 16 a that is composed of a plurality of optical elements; a focuslens 21 that serves as a focus switching mechanism so as to selectivelyfocus on two observation areas for distant observation and closeobservation; and an actuator 22 that drives the focus lens 21.

The image acquisition unit 19 is provided closer to a rear-end portionof the insertion portion 6 than the objective optical system 16 is andis provided with: a polarizing beam splitter 18 (optical-path dividingmeans) that divides a subject image into two optical images havingdifferent focus positions; and an image acquisition element 17 thatimages the two optical images to acquire two images.

As shown in FIG. 2, the polarizing beam splitter 18 is provided with adepolarizing plate 28, a first prism 18 a, a second prism 18 b, a mirror18 c, and a λ/4 wavelength plate 18 d.

A λ/4 wavelength plate is used as the depolarizing plate 28. If ageneral depolarizing plate (for producing non-polarized light throughscrambling) is used in order to change the polarized state to anon-polarized state, because the size thereof is too large, and thestructure thereof is complicated, the depolarizing plate is not suitablefor being disposed in the distal end of the endoscope. Thus, a λ/4wavelength plate is used as the depolarizing plate 28, thereby making itpossible to convert polarized light entering the polarizing beamsplitter 18 into substantially circularly polarized light and to achieveintensity uniformity.

Usually, a λ/4 wavelength plate needs to be considered at zero order,and, for example, when a wavelength plate using a crystal is used togive a ¼λ phase difference at a design wavelength of 550 nm, it isnecessary to reduce the thickness of the crystal to about 15 μm.However, it is impractical to directly handle such an ultrathin filter.For the purpose of increasing the plate thickness to a practical level,a multiple-order wavelength plate is designed so as to obtain apredetermined phase difference at higher order. For example, when aphase difference of 2.5 wavelengths is produced at the wavelength of 550nm, the plate thickness can be increased to about 150 μm if a crystal isused.

The phase difference of 2.5 wavelengths can be considered as effectivelya phase difference of 0.25 wavelengths (=¼). However, with the increasedplate thickness, an unignorable phase-difference shift is caused even bya slight wavelength shift or an oblique incident light beam; therefore,an image-compositing processing unit 33, to be described later,appropriately sets compositing ratios (mask) that are used to compositea pair of images, thereby making it possible to generate a good imagehaving less luminance unevenness.

The first prism 18 a and the second prism 18 b both have beam splittingsurfaces that are inclined at an angle of 45 degrees with respect to theoptical axis. The beam splitting surface of the first prism 18 a isprovided with a polarized-light separating film 18 e. Then, the firstprism 18 a and the second prism 18 b constitute the polarizing beamsplitter 18 by bringing their beam splitting surfaces into contact, withthe polarized-light separating film 18 e therebetween. Furthermore, themirror 18 c is provided in the vicinity of an end surface of the firstprism 18 a via the λ/4 wavelength plate 18 d, and the image acquisitionelement 17 is attached to an end surface of the second prism 18 b. Notethat the λ/4 wavelength plate 18 d has inverse dispersion properties.

By making the λ/4 wavelength plate 18 d have the inverse dispersionproperties, it is possible to provide an endoscope in which afluctuation in the extinction ratio is reduced in visible wavelengthbands, thus suppressing the luminance unevenness.

The reason is as follows. Specifically, it is preferred that two imagesobtained through polarized-light separation at the polarizing beamsplitter 18 do not have a large difference in luminance intensity. In areturning light path, polarized light passing through the λ/4 wavelengthplate and the mirror is rotated in direction by 90 degrees, thusbecoming P-polarized light. The light becoming the P-polarized light istransmitted through the polarizing beam splitter 18 and is received bythe image acquisition element. In other words, if a normal-dispersionwaveplate, which has been conventionally used as a λ/4 wavelength plate,is used, light that is normally rotated by 90 degrees to becomeP-polarized light is under rotated or over rotated depending on thewavelength, thus causing the extinction ratio to fluctuate whentransmitted through the polarizing beam splitter 18 and thus causingluminance unevenness.

This is not a problem for pickup optical systems or laser targetingdevices as long as a suitable wavelength plate for a wavelength band tobe used is selected, but it is fatal when observation using a visiblewavelength band and a narrow band is performed, as in endoscopes.

A subject image from the objective optical system 16 enters thedepolarizing plate 28, and polarized light produced when entering atBrewster's angle is depolarized by the depolarizing plate 28 and isseparated in the first prism 18 a into a P-component (transmitted light)and an S-component (reflected light) by the polarized-light separatingfilm 18 e, which is provided on the beam splitting surface of the firstprism 18 a. Thus, the subject image is separated into two opticalimages, i.e., an optical image of the reflected light and an opticalimage of the transmitted light.

The S-component optical image is reflected at the polarized-lightseparating film 18 e toward the opposite side to the image acquisitionelement 17, travels in a light path A, is transmitted through the λ/4wavelength plate 18 d, and is returned at the mirror 18 c toward theimage acquisition element 17. The returned optical image is againtransmitted through the λ/4 wavelength plate 18 d, thus being rotated inits polarized-light direction by 90 degrees, is transmitted through thepolarized-light separating film 18 e, and is imaged on the imageacquisition element 17.

The P-component optical image is transmitted through the polarized-lightseparating film 18 e, travels in a light path B, is reflected at amirror surface that reflects the image perpendicularly toward the imageacquisition element 17 and that is provided at an opposite side of thesecond prism 18 b to the beam splitting surface thereof, and is imagedon the image acquisition element 17. At this time, a prism glass path isset so as to cause a predetermined optical path difference of aboutseveral tens of μm, for example, between the light path A and the lightpath B, and two optical images having different focus positions areimaged on a light-receiving surface of the image acquisition element 17.Accordingly, it is possible to obtain, as a pair of images, a near pointimage (image A) through the light path A and a far point image (image B)through the light path B.

Specifically, in order to be able to separate the subject image into twooptical images having different focus positions, the first prism 18 aand the second prism 18 b are disposed such that the optical path lengthon the reflected light side becomes shorter (smaller) than the opticalpath length (glass path length) on the transmitted light side leading tothe image acquisition element 17, in the first prism 18 a.

As shown in FIG. 3, in order to acquire two optical images havingdifferent focus positions by individually receiving the light beams, theimage acquisition element 17 is provided with two light-receiving areas(effective pixel areas) 17 a and 17 b in the entire pixel area of theimage acquisition element 17. In order to acquire two optical images,the light-receiving areas 17 a and 17 b are arranged so as to correspondwith imaging planes of these optical images, respectively. Then, in theimage acquisition element 17, the light-receiving area 17 a is shiftedin focus position toward a near point side relative to thelight-receiving area 17 b, and the light-receiving area 17 b is shiftedin focus position toward a far point side relative to thelight-receiving area 17 a. Accordingly, two optical images havingdifferent focus positions are imaged on the light-receiving surface ofthe image acquisition element 17.

Furthermore, a correction pixel area 17 c that is used to correct ageometric shift between the two separated optical images is providedaround the light-receiving areas 17 a and 17 b. Manufacturing errors arecorrected in the correction pixel area 17 c, and correction is performedthrough image processing by an image-correction processing unit 32, tobe described later, thereby resolving the above-described optical-imagegeometric shift.

Note that a description has been given of an example case in which a λ/4wavelength plate is used as the depolarizing plate 28; however, thedepolarizing plate is not limited thereto, and a zero-order λ/4wavelength plate that is connected to a glass or quartz substratethrough bonding or optical contact, shown in FIG. 4A, can be appliedthereto. Even with a zero-order λ/4 wavelength plate, which has toosmall a thickness to be used by itself, when the zero-order λ/4wavelength plate is bonded to a glass substrate or a quartz substrate, adesired strength can be maintained. For example, when a crystal is usedfor the depolarizing plate 28, the thickness thereof becomes 15 μm,which is very thin, at the design wavelength of 550 nm, thus makinghandling extremely difficult. Thus, the crystal is bonded to a glasssubstrate or a quartz substrate, thus maintaining a desired strength andmaking handling easy.

Furthermore, as shown in FIG. 4B, it is also possible to use, as thedepolarizing plate 28, two inorganic wavelength plates whose opticalaxes are perpendicular to each other and to configure them such that thethickness t (mm) of at least one of the two inorganic wavelength platessatisfies the following conditional expressions,

0.135≦t≦0.3  (1)

t=(k*0.25λ/Δn)*10⁻⁶  (2)

9≦k≦19.5  (3)

where t (mm) is the thickness of a single inorganic wavelength plate, kis a desired coefficient, λ (nm) is a design center wavelength, and Δnis a difference between a refractive index at ordinary light and arefractive index at extraordinary light.

Although the cost is increased when a zero-order wavelength plate isused, a multiple-order wavelength plate made of the same material isused in this way, thereby making is possible to realize a functionsubstantially equivalent to that of a zero-order wavelength plate, whilereducing the cost. Specifically, a depolarizing plate with low angledependence and high performance can be obtained.

The above-described conditional expressions define the thickness of asingle wavelength plate for allowing easy handling and for preventing anincrease in the size of the endoscope distal-end portion. Furthermore,when wavelength plates made of the same material are used, the thicknessdifference Δt therebetween is configured so as to obtain a desired phasedifference. When a depolarizing plate formed of two wavelength plates isused, it is preferred that the total thickness of the depolarizing platefall within the range from 0.3 to 0.6 mm in order to prevent an increasein the size of the endoscope distal-end portion and to be resistant toimpacts or stress.

Conceivable specific examples of inorganic wavelength plates include aconfiguration in which two inorganic wavelength plates are both made ofcrystal and a configuration in which the two inorganic wavelength platesare made of a combination of different materials, i.e., crystal andMgF₂.

Furthermore, as shown in FIG. 4C, the depolarizing plate 28 can beconfigured by sandwiching a polymer film between glass or quartzsubstrates.

FIG. 5A is a graph showing the intensities of a pair of images when awavelength plate having a thickness t of 0.15 mm is used as adepolarizing plate. Furthermore, FIG. 5B is a graph showing theintensities of a pair of images when a wavelength plate having athickness t of 0.27 mm is used as a depolarizing plate.

FIG. 6 shows wavelength dispersion properties when the depolarizingplate 28 and the λ/4 wavelength plate 18 d have inverse dispersionproperties. Note that, in FIG. 6, a two-dot chain line indicates adepolarizing plate using inorganic wavelength plates made of acombination of different materials, and a broken line indicates a λ/4wavelength plate that is formed of a polymer film.

Furthermore, a polymer film can be used as the λ/4 wavelength plate 18d, and, in this case, it is possible to contribute to a reduction in thesize of the endoscope system and to reduce the cost. Furthermore, theabove-described λ/4 wavelength plate 18 d has inverse dispersionproperties; however, the λ/4 wavelength plate 18 d need not necessarilyhave inverse dispersion properties if a zero-order wavelength plate isused as the λ/4 wavelength plate 18 d, for example. FIG. 7 shows anintensity distribution graph of a pair of images when a zero-orderwavelength plate that gives a λ/4 phase difference at the wavelength of550 nm is used, for example.

The focus lens 21 can be moved to two positions in the optical axisdirection and is driven by the actuator 22 so as to be moved from oneposition to another position and from the other position to the oneposition between the two positions. When the focus lens 21 is set at thefront-side (object-side) position, a subject located in an observationarea used for distant observation is set to be focused on, and when thefocus lens 21 is set at the rear-side position, a subject located in anobservation area used for close observation is set to be focused on.

Note that the actuator 22 is connected to a signal line 23 insertedthrough the insertion portion 6, and the signal line 23 is furtherinserted through a second cable 24 extending from the operating portion7. A signal connector 24 a provided at an end portion of the secondcable 24 is detachably connected to the processor device 4, and thesignal line 23 is connected to an actuator control unit 25 that isprovided in the processor device 4.

The actuator control unit 25 receives, for example, a switchingoperation signal from a switching operation switch 26 that is providedin the operating portion 7 of the endoscope 2. The actuator control unit25 applies a driving signal for electrically driving the actuator 22, inresponse to the operation of the switching operation switch 26, to movethe focus lens 21.

Note that the switching operation means for generating a switchingoperation signal is not limited to the switching operation switch 26 andmay be a switching operation lever or the like. The focus lens 21, theactuator 22, and the actuator control unit 25 constitute a focusswitching mechanism. Incidentally, a focus means in this application isnot limited to the above-described means for moving the focus lens inthe optical-axis direction. For example, it is also possible to adopt ameans for switching focus by inserting or detaching a lens or a filterinto or from the objective optical system.

The image acquisition element 17 is connected to signal lines 27 a thatis inserted through the insertion portion 6, the operating portion 7,and the second cable 24 and is connected to an image processor 30 thatis provided in the processor device 4 and that serves as an imageprocessing unit, when the signal connector 24 a is connected to theprocessor device 4.

The image processor 30 is provided with: an image reading unit 31 thatreads respective images of two optical images having different focuspositions, acquired by the image acquisition element 17; theimage-correction processing unit 32, which performs image correction forthe two images read by the image reading unit 31; and theimage-compositing processing unit 33, which performs image compositingprocessing for compositing the two corrected images.

The image-correction processing unit 32 corrects the images of the twooptical images, which are imaged on the light-receiving areas 17 a and17 b of the image acquisition element 17, such that differences otherthan the difference in their mutual focus positions become almost thesame. Specifically, the two images are corrected such that the relativepositions, the angles, and the magnifications in the optical images ofthe two images become almost the same. Specifically, theimage-correction processing unit 32 uses the correction pixel area 17 cto perform correction such that the relative position, the angle, andthe magnification in the light-receiving area 17 a become almost thesame as those in the light-receiving area 17 b with reference to thelight-receiving area 17 b, for example.

In a case in which a subject image is separated, and the separatedimages are individually imaged on the image acquisition element 17,geometric differences therebetween may be caused in some cases.Specifically, the respective optical images imaged on thelight-receiving areas 17 a and 17 b of the image acquisition element 17may have a relative shift in magnification, shift in position, or shiftin angle, i.e., in rotational direction, in some cases. Although it isdifficult to completely resolve these differences at the time ofmanufacturing etc., if the amounts of the shifts are increased, acomposited image may become a double image, or an unnatural luminanceunevenness etc. may be caused therein. Thus, the image-correctionprocessing unit 32 corrects the above-described geometric differencesand luminance difference.

In order to correct the luminance difference between two images, it ispreferred that correction be performed with reference to an opticalimage or image that has a higher luminance of two optical images orimages or an optical image or image that has a higher luminance atrelatively the same position of the two optical images or images.

In other words, luminance correction is performed so as to conform to arelatively brighter image, i.e., an image having a higher intensity(signal level) of a luminance signal (for example, G signal). If abrighter image is corrected by being multiplied by a negative gain,color saturation noise is likely to occur. In particular, in an imagehaving an area that is saturated by causing halation, there is a problemin that color unevenness is caused in the halation area. This is becauseRGB sensitivities at the pixels in a sensor are different. In a state ofhalation or high luminance close thereto, because the state is alreadysaturated even if an attempt to reduce the gain is made, for example, Rcannot be corrected, and only gains for G and B are reduced, and, as aresult, the halation portion becomes magenta colored.

The image-compositing processing unit 33 generates a composited image bycompositing the corresponding pixels between the pair of imagescorrected by the image-correction processing unit 32 and is providedwith: a contrast calculating unit 41, a compositing-mask generating unit42, a mask storing unit 43, a mask correcting unit 44, and an imagecompositing unit 45, as shown in FIG. 8.

The contrast calculating unit 41 calculates the contrast imageacquisition element 17 and corrected by the image-correction processingunit 32. In other words, in each of the pair of images, a frequencycomponent is calculated, and the contrast is calculated, from thisfrequency component, to serve as an evaluation value indicating thedegree of in-focus at each pixel. In calculating the frequencycomponent, it is possible to use an existing differential filter, aband-pass filter designed for a subject, or the like.

The compositing-mask generating unit 42 calculates, from the contrastscalculated in the contrast calculating unit 41, contrast ratios, whichare the ratios of the contrasts, and generates a compositing mask thatserves as the compositing ratios of the corresponding pixels between thepair of images, on the basis of the calculated contrast ratios. Thegenerated compositing mask is stored in the mask storing unit 43.Furthermore, the mask storing unit 43 stores a plurality of compositingmasks generated in time series by the compositing-mask generating unit42 with respect to a plurality of pairs of images acquired in timeseries by the image acquisition element 17. Note that it is preferredthat, when a mask is corrected by the mask correcting unit 44, to bedescribed later, the corrected mask be stored in the mask storing unit43 instead of the compositing mask.

The mask correcting unit 44 refers to a plurality of compositing masksthat have already been generated for the past images and stored in themask storing unit 43, corrects the compositing mask, and generates acorrected mask. Specifically, the mask correcting unit 44 performsweighting such that the percentage of past compositing masks is higherat pixels that constitute a static area and an area having contrastlower than a predetermined threshold, in the pair of images, than atpixels that constitute a moving-object area or an area having contrastequal to or higher than the predetermined threshold, in the pair ofimages, and subjects the plurality of compositing masks to weightedaveraging, thereby generating a corrected mask. Note that determinationof a moving-object area is performed as follows. In other words, as aresult of comparing the past masks stored in the mask storing unit 43with the calculated compositing mask, an area with a large difference isdetermined as an area with a large change from the past masks, i.e., asa moving-object area.

The image compositing unit 45 generates a composited image bycompositing the pair of images according to the corrected mask.

Furthermore, the image processor 30 has: a post image processing unit 34that subjects the composited image generated by the image-compositingprocessing unit 33 to post image processing, such as color matrixprocessing, contour enhancement, and gamma correction; and an imageoutput unit 35 that outputs an image that has been subjected to the postimage processing. The image output from the image output unit 35 isoutput to the image display device 5.

Furthermore, the image processor 30 has a light control unit 36 thatgenerates, from the images read by the image reading unit 31, a lightcontrol signal for controlling light to obtain the standard luminanceand outputs the light control signal generated by the light control unit36 to the diaphragm driving unit 14 of the light source device 3. Thediaphragm driving unit 14 adjusts, according to the light controlsignal, the degree of opening of the diaphragm 12 so as to maintain thestandard luminance.

Furthermore, in this embodiment, a correction-parameter storing unit 37that stores (information of) correction parameters that are used tocorrect images in the image-correction processing unit 32 is provided.

Here, a correction parameter is determined in consideration of theproperties of an optical-path dividing element, an image acquisitionelement, and a λ/4 plate. For example, depending on the shadingproperties of an optical-path dividing element and an image acquisitionelement and the wavelength characteristics of a λ/4 wavelength plate,the above-described geometric differences and luminance difference or acolor difference may be caused in the images of the two optical images.

Furthermore, when the wavelength band of illumination light produced inthe light source device is selectively switched, for example, when theendoscope system has a plurality of observation modes, such as a normalobservation mode, a narrow-band observation mode, etc., and uses adifferent wavelength band of illumination light depending on theobservation mode, a luminance difference may also be caused between thetwo images. If there is a luminance difference between the two images,unnatural luminance unevenness or color unevenness is caused in acomposited image.

Thus, for example, in consideration of the properties of theoptical-path dividing element, the image acquisition element, and theλ/4 plate, luminance correction parameters corresponding to theobservation modes, i.e., corresponding to the wavelength bands ofillumination light, are stored in the correction-parameter storing unit37 as IDs for the respective observation modes. A control unit 39switches the correction parameter corresponding to the observation mode,and the image-correction processing unit 32 performs image correction onthe basis of the selected correction parameter.

More specifically, for example, when the wavelength dependence of a ¼λwavelength plate or a depolarizing plate is large (when an achromat isnot used), or when the polarized-light separating film of the polarizingbeam splitter has large wavelength dependence, the luminance correctionvalue does not necessarily become optimum depending on the wavelengthband of illumination light for observing a subject, thus causingluminance unevenness, in some cases.

FIGS. 9 to 11 are graphs showing the spectral characteristics of apolarization separation coating used for the polarizing beam splitter.In each figure, a solid line indicates a spectral graph (Tp) of thefar-point light path, and a broken line indicates a spectral graph(Tp*Rs/100) of the near-point light path.

FIG. 9 shows the spectral characteristics of a light beam entering, atan angle of 0 degrees, the polarizing-beam-splitter surface, which isinclined at an angle of 45 degrees with respect to the optical axis,i.e., shows the spectral characteristics on the optical axis. FIG. 10shows the spectral characteristics when a light beam inclined at anangle of −7 degrees with respect to the optical axis enters thepolarization separation coating. FIG. 11 shows the spectralcharacteristics when a light beam inclined at an angle of +7 degreesenters the polarization separation coating. Note that the sign of theangle is positive in the clockwise direction.

A description will be given below of an example case in which a lightsource using a frame sequential method is used as the light sourcedevice 3 of this embodiment, and there are three observation modes,i.e., a normal observation mode, a narrow-band observation mode 1, and anarrow-band observation mode 2. In this case, as wavelength bands usedfor RGB in the normal observation mode, a wavelength band from 600 to650 nm is used for R, a wavelength band from 510 to 580 nm is used forG, and a wavelength band from 450 to 480 nm is used for B. On the otherhand, in the narrow-band observation mode 1, which mainly uses a shortwavelength side, a wavelength band from 525 to 550 nm is used for G, anda wavelength band from 400 to 430 nm is used for B.

As shown in FIGS. 10 and 11, it is understood that there is a largedifference, in particular, in the intensity of B in each of theoblique-incidence observation modes. In other words, if the samecorrection parameter is used for images acquired in the normalobservation mode and images acquired in the narrow-band observation mode1, luminance unevenness is caused in the images acquired in thenarrow-band observation mode 1. Therefore, it is necessary to applycorrection parameters corresponding to the respective observation modesby making a luminance-correction parameter for images acquired in thenormal observation mode different from a luminance-correction parameterfor images acquired in the narrow-band observation mode 1.

Thus, the correction parameter is switched between the case where thewavelength band from 450 to 480 nm for B is used in the normalobservation mode and the case where the wavelength band from 400 to 430nm for B is used in the narrow-band observation mode 1.

Furthermore, in the narrow-band observation mode 2, a wavelength bandfrom 590 to 610 nm is used for R1, a wavelength band from 620 to 640 nmis used for R2, and a wavelength band from 450 to 470 nm is used for B.Specifically, the narrow-band observation mode 2 is an observation modein which a wavelength band from 510 nm to 580 nm is not used.

In this case, in the normal observation mode, a correction parameterthat is determined in consideration of a wavelength band from 525 to 550nm for G, too, is used; thus, when the observation mode is switched fromthe normal observation mode to the narrow-band observation mode 2, a gapin luminance correction parameter is caused, thus causing luminanceunevenness.

Thus, when the wavelength band from 525 to 550 nm for G, which is notused in the narrow-band observation mode 2 and is used in the normalobservation mode, is used, the correction parameter is switched.

Switching of the correction parameter can be interlocked with switchingof the observation mode, i.e., can be interlocked with anobservation-mode switch SW on an endoscope body, a foot SW (not shown),or the like.

For example, when the observation-mode switch SW is pressed to switchthe observation mode, a filter is switched in the light source device onreceiving a switch signal, thereby radiating light having a wavelengthband corresponding to the selected observation mode. Then, inconjunction with switching of the observation mode, a correctionparameter corresponding to the selected observation mode is read fromthe correction-parameter storing unit 37, and the optimum correctionprocessing is performed in the image-correction processing unit 32. Notethat, if the light source is an LED light source, it is possible toswitch to an LED corresponding to the selected observation mode, and theparameter may be switched in conjunction therewith.

Next, the flow of compositing a pair of optical images in thisembodiment will be described with reference to a flowchart shown in FIG.12. Note that a description will be given below of an example case inwhich three past compositing masks are used when a compositing mask iscorrected.

In Step S1, counting is started to count the number of times that a pastmask required for compositing-mask correction is generated. In Step S2,the image acquisition element 17 acquires, for one subject, the farpoint image of a far-point optical image and the near point image of anear-point optical image that have different focus positions. In StepS3, the image-correction processing unit 32 subjects the far point imageand the near point image, which are acquired in Step S2, to correctionprocessing. Specifically, the two images are corrected, according to acorrection parameter set in advance, such that the relative positions,the angles, and the magnifications in the images become almost the same,and the corrected images are output to the image-compositing processingunit 33. Note that the two images may be subjected to correction interms of luminance or color difference as needed.

In Step S4, the contrast calculating unit 41 of the image-compositingprocessing unit 33 calculates the contrast values of the pixels of thetwo images that have been subjected to correction processing. Thecalculated contrast values are output to the compositing-mask generatingunit 42. In Step S5, from the contrasts calculated in the contrastcalculating unit 41, the contrast ratios, which are the ratios of thecontrasts, are calculated. A compositing mask that serves as compositingratios of corresponding pixels between the pair of images is generatedon the basis of the calculated contrast ratios and is stored in the maskstoring unit 43.

In Step S6, it is determined whether processing including: acquisitionof a pair of images in the image acquisition element 17; thepredetermined correction processing; calculation of the contrasts; andgeneration and storing of the compositing mask has been performed fourtimes. If the processing has not been performed four times, the flowreturns to Step S2. If the processing has been performed four times, theflow advances to Step S7.

In Step S7, the mask correcting unit 44 determines weights such that thepercentage of the past compositing masks is higher at pixels thatconstitute a static area and an area having contrast lower than apredetermined threshold, in the pair of images, than at pixels thatconstitute a moving-object area and an area having contrast equal to orhigher than the predetermined threshold, in the pair of images. Here,the weights can be sequentially calculated according to a predeterminedconditional expression, for example, or can be determined in advance, asshown in FIG. 13.

In Step S8, a corrected mask is generated by correcting the compositingmask according to the weights determined in Step S7 and is output to theimage compositing unit 45. In Step S9, a composited image is generatedby compositing the pair of images including the far point image and thenear point image according to the corrected mask. In Step S10, it isdetermined whether the image compositing processing has been finished.If the image compositing processing has not been finished, in Step S11,the counting numbers of compositing masks stored in the mask storingunit 43 are sequentially shifted (see FIG. 14), n is set to 4, andprocessing from Step S2 to Step S5 is performed.

Note that a description has been given above of an example case in whichfour compositing masks are subjected to weighted averaging; however, thepresent invention is not limited thereto, and the number of masks to besubjected to weighted averaging and the weights thereof can bedetermined as appropriate.

In this way, according to this embodiment, it is possible to acquire animage in which the depth of field is extended while suppressingluminance unevenness caused by spectral intensity, by preventing adiscontinuous area from being caused in a composited image due to noiseetc. and further by optimizing the depolarizing plate and the λ/4wavelength plate. Furthermore, in consideration of a case in which thecompositing mask varies over time, e.g., a case in which a moving-objectarea is included in the image, a plurality of compositing masksgenerated in time series are subjected to weighted averaging to generatea corrected mask; therefore, it is possible to prevent a fluctuation inthe compositing ratios applied to each frame, in particular, in a lowcontrast area etc, when images are acquired in time series as in amoving image. Specifically, it is possible to prevent the occurrence offlickering or luminance unevenness that would be caused by switching,for each frame, between an image to which the pixels of a far pointimage are applied and an image to which the pixels of a near point imageare applied, despite the pixel positions being the same.

The above-described embodiment leads to the following inventions.

According to one aspect, the present invention provides an endoscopesystem including: an objective optical system that is provided at adistal end of an insertion portion and that obtains a subject image of asubject irradiated with illumination light from a light source; anoptical-path dividing means that divides the subject image into twooptical images having different focus positions; an image acquisitionelement that simultaneously images the two optical images havingdifferent focus positions to acquire a pair of images; a contrastcalculating means that calculates contrasts, for respective pixels, ofthe pair of images acquired by the image acquisition element; a maskgenerating means that generates a compositing mask that serves ascompositing ratios of the corresponding pixels between the pair ofimages on the basis of the ratios of the contrasts calculated by thecontrast calculating means; a mask correcting means that generates acorrected mask by applying, for the respective pixels, weightedaveraging to a plurality of compositing masks that are generated in timeseries by the mask generating means for a plurality of pairs of imagesthat are acquired in time series by the image acquisition element; andan image compositing means that composites the two images according tothe corrected mask generated by the mask correcting means, wherein themask correcting means applies weighted averaging to the plurality ofcompositing masks by performing weighting such that the percentage ofthe past compositing masks is higher at pixels that constitute a staticarea and an area having contrast lower than a predetermined threshold,in the pair of images, than at pixels that constitute a moving-objectarea or an area having contrast equal to or higher than thepredetermined threshold, in the pair of images.

According to the above-described aspect, when a composited image isgenerated from a pair of images that are acquired by simultaneouslyimaging two optical images having different focus positions, thecontrast calculating means calculates the contrasts for respectivepixels of the pair of images. Here, the contrast is used as anevaluation value indicating whether a certain pixel is in focus, and ifthe contrast is higher, it can be determined that the pixel is in focus.Thus, by calculating the contrasts of the pair of images, it can bedetermined that a pixel in question is in focus at a far point, at anear point, or at an intermediate point.

Therefore, in the compositing mask generated by the mask generatingmeans on the basis of the ratios of the contrasts for the respectivepixels, the compositing ratio is set higher at a pixel having highercontrast of the corresponding pixels between the pair of images.

In this case, in consideration of a case in which the compositing maskvaries over time, e.g., a case in which a moving-object area is includedin the image, the mask correcting means subjects a plurality ofcompositing masks generated in time series by the mask generating meansto weighted averaging, thus generating a corrected mask. Specifically,the mask correcting means generates a corrected mask in which thepercentage of the relatively new past compositing masks is higher for astatic area and an area having contrast lower than a predeterminedthreshold and existing in the pair of images than for a moving-objectarea or an area having contrast equal to or higher than thepredetermined threshold and existing in the pair of images.

By doing so, in a case in which images are acquired in time series as ina moving image, in particular, in a low contrast area etc., thecompositing ratios applied to each frame can be prevented from beingfluctuated. Specifically, it is possible to prevent the occurrence offlickering or luminance unevenness caused by switching, for each frame,between an image to which the pixels of a far point image are appliedand an image to which the pixels of a near point image are applied,despite the pixel positions being the same.

In the above-described aspect, it is preferred to further include: adepolarizing plate that is formed of at least one wavelength platedisposed between the objective optical system and the optical-pathdividing means; and a λ/4 wavelength plate and a reflective mirror thatare disposed in the light path of one optical image of the two opticalimages.

By doing so, polarized light produced when entering at Brewster's anglecan be depolarized by the depolarizing plate, and the phase of the lightcan be rotated by making the light pass through the optical-pathdividing means.

In the above-described aspect, it is preferred that the A/4 wavelengthplate have inverse dispersion properties.

By doing so, it is possible to prevent the occurrence of unevennesscaused by the difference in intensity at the observation wavelength.

In the above-described aspect, it is preferred that the λ/4 wavelengthplate be a polymer film.

By doing so, it is possible to contribute to a reduction in the size ofthe endoscope system and to reduce the cost.

In the above-described aspect, it is preferred that the λ/4 wavelengthplate be a single zero-order wavelength plate.

By doing so, it is possible to obtain a depolarizing plate with lowangle dependence and high performance and to contribute to a reductionin the size of the endoscope system.

In the above-described aspect, it is preferred that the depolarizingplate be a zero-order λ/4 wavelength plate that is connected to a glassor quartz substrate through bonding or optical contact.

By doing so, it is possible to obtain a depolarizing plate with lowangle dependence and high performance and to contribute to a reductionin the size of the endoscope system.

In the above-described aspect, it is preferred that the depolarizingplate be formed of two inorganic wavelength plates whose optical axesare perpendicular to each other, and the thickness t (mm) of at leastone of the two inorganic wavelength plates satisfy the followingconditional expressions,

0.135≦t≦0.3  (1)

t=(k*0.25λ/Δn)*10̂−6  (2)

9≦k≦19.5  (3)

where t (mm) is the thickness of a single inorganic wavelength plate, kis a desired coefficient, λ (nm) is a design center wavelength, and Δnis a difference between a refractive index at ordinary light and arefractive index at extraordinary light.

By doing so, it is possible to obtain a depolarizing plate with lowangle dependence and high performance while reducing the cost.

In the above-described aspect, it is preferred that refractive indexesof the two inorganic wavelength plates be different from each other.

By doing so, it is possible to convert polarized light corresponding tolinear polarized light into circularly polarized light, in considerationof wavelength dependent properties.

In the above-described aspect, it is preferred that the depolarizingplate be a polymer film that is sandwiched between glass or quartzsubstrates.

By doing so, angle dependence can be reduced, and polarized light can beeasily depolarized.

It is preferred to further include an image correcting means thatcorrects a luminance difference between the pair of images on the basisof a predetermined correction parameter, wherein the light sourceselectively switches between illumination light having wavelength bandsdifferent from each other to radiate illumination light having theselected wavelength band onto the subject; and, when the imageacquisition element acquires a pair of images based on a subject imageof the subject irradiated with illumination light having the selectedwavelength band, the correction parameter is switched according to thewavelength band of the illumination light when the image correctingmeans performs correction.

By doing so, even when a luminance difference is caused between a pairof images by the wavelength band of illumination light produced by thelight source, correction is performed according to the wavelength bandof illumination light, thereby making it possible to suppress theoccurrence of unnatural luminance unevenness or color unevenness in acomposited image.

The light source may selectively switch between illumination light thatincludes a wavelength band from 400 to 430 nm and illumination lightthat includes a wavelength band from 450 to 480 nm and may radiate theselected illumination light.

Furthermore, the light source may selectively switch to illuminationlight that does not include a wavelength band from 510 to 580 nm.

By doing so, correction according to the wavelength band is performedfor images based on illumination light including a wavelength band thattends to cause luminance unevenness, thereby making it possible tosuppress unnatural luminance unevenness or color unevenness.

REFERENCE SIGNS LIST

-   1 endoscope system-   2 endoscope-   3 light source device-   4 processor device-   5 image display device-   6 insertion portion-   16 objective optical system-   17 image acquisition element-   17 a, 17 b light-receiving areas-   17 c correction pixel area-   18 polarizing beam splitter-   18 a first prism-   18 b second prism-   18 c mirror-   18 d λ/4 wavelength plate-   18 e polarized-light separating film-   19 image acquisition unit-   28 depolarizing plate-   30 image processor-   32 image-correction processing unit-   33 image-compositing processing unit-   41 contrast calculating unit-   42 compositing-mask generating unit-   43 mask storing unit-   44 mask correcting unit-   45 image compositing unit

1. An endoscope system comprising: an objective optical system that isprovided at a distal end of an insertion portion and that obtains asubject image of a subject irradiated with illumination light from alight source; an optical-path dividing portion that divides the subjectimage into two optical images having different focus positions; an imageacquisition element that simultaneously images the two optical imageshaving different focus positions to acquire a pair of images; a contrastcalculating portion that calculates contrasts, for respective pixels, ofthe pair of images acquired by the image acquisition element; a maskgenerating portion that generates a compositing mask that serves ascompositing ratios of the corresponding pixels between the pair ofimages on the basis of the ratios of the contrasts calculated by thecontrast calculating portion; a mask correcting portion that generates acorrected mask by applying, for the respective pixels, weightedaveraging to a plurality of compositing masks that are generated in timeseries by the mask generating portion for a plurality of pairs of imagesthat are acquired in time series by the image acquisition element; andan image compositing portion that composites the two images according tothe corrected mask generated by the mask correcting portion, wherein themask correcting portion applies weighted averaging to the plurality ofcompositing masks by performing weighting such that the percentage ofthe past compositing masks is higher at pixels that constitute a staticarea and an area having contrast lower than a predetermined threshold,in the pair of images, than at pixels that constitute a moving-objectarea or an area having contrast equal to or higher than thepredetermined threshold, in the pair of images.
 2. An endoscope systemaccording to claim 1, further comprising: a depolarizing plate that isformed of at least one wavelength plate disposed between the objectiveoptical system and the optical-path dividing portion; and a λ/4wavelength plate and a reflective mirror that are disposed in the lightpath of one optical image of the two optical images.
 3. An endoscopesystem according to claim 2, wherein the λ/4 wavelength plate hasinverse dispersion properties.
 4. An endoscope system according to claim3, wherein the λ/4 wavelength plate is a polymer film.
 5. An endoscopesystem according to claim 2, wherein the λ/4 wavelength plate is asingle zero-order wavelength plate.
 6. An endoscope system according toclaim 2, wherein the depolarizing plate is a zero-order λ/4 wavelengthplate that is connected to a glass or quartz substrate through bondingor optical contact.
 7. An endoscope system according to claim 2, whereinthe depolarizing plate is formed of two inorganic wavelength plateswhose optical axes are perpendicular to each other, and the thickness t(mm) of at least one of the two inorganic wavelength plates satisfiesthe following conditional expressions,0.135≦t≦0.3  (1)t=(k*0.25λ/Δn)*10̂−6  (2)9≦k≦19.5  (3) where t (mm) is the thickness of a single inorganicwavelength plate, k is a desired coefficient, λ (nm) is a design centerwavelength, and Δn is a difference between a refractive index atordinary light and a refractive index at extraordinary light.
 8. Anendoscope system according to claim 7, wherein refractive indexes of thetwo inorganic wavelength plates are different from each other.
 9. Anendoscope system according to claim 2, wherein the depolarizing plate isa polymer film that is sandwiched between glass or quartz substrates.10. An endoscope system according to claim 1, further comprising animage correcting portion that corrects a luminance difference betweenthe pair of images on the basis of a predetermined correction parameter,wherein the light source selectively switches between illumination lighthaving wavelength bands different from each other to radiateillumination light having the selected wavelength band onto the subject;and when the image acquisition element acquires a pair of images basedon a subject image of the subject irradiated with illumination lighthaving the selected wavelength band, the correction parameter isswitched according to the wavelength band of the illumination light whenthe image correcting portion performs correction.
 11. An endoscopesystem according to claim 10, wherein the light source selectivelyswitches between illumination light that includes a wavelength band from400 to 430 nm and illumination light that includes a wavelength bandfrom 450 to 480 nm and radiates the selected illumination light.
 12. Anendoscope system according to claim 10, wherein the light sourceselectively switches to illumination light that does not include awavelength band from 510 to 580 nm.